Eyetrack-driven illumination and information display

ABSTRACT

An eyetracker is used to control power to an electrical device such as a computer display screen so that power consumption is reduced when a user&#39;s eyes and therefore a user&#39;s attention are not directed to the device. A motion detector activates a proximity detector and/or an IR detector to ensure that power is applied only when a user is actually present.

This application is a division of application Ser. No. 08/655,134 filedMay 30, 1996, now U.S. Pat. No. 5,835,083.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates to controlling power consumption in electricaldevices having a finite source of energy, such as battery driven devicesand, more particularly, to controlling power consumption in the use ofcomputer displays.

2. Description of Related Art

Computer displays face a problem that if some types of display are leftrunning for a long period of time with a particular image formed on thescreen, each portion of the image formed on the screen would become moreor less permanently etched into the screen. CRT displays areparticularly susceptible to this problem.

Screen savers were developed to minimize this type of occurrence. Ascreen saver program is loaded and activated if a period of time elapsesduring which nothing is typed or no mouse movement is detected. Inshort, when a computer is left unattended with no activity, a screensaver would blacken the screen totally, except, perhaps, for a movingdisplay which would cross the screen in an irregular pattern so that thesame pattern would not be displayed at the same location on the screenfor extended periods of time.

Flat panel displays, and other types of display also utilize screensaver programs in part to equalize the on and off times of drivercircuit elements so that certain driver circuits or light emittingelements were not utilized substantially more than others, aging morerapidly and becoming thus more prone to failure.

Eyetracking devices are known particularly in conjunction with heads updisplays in certain control applications in aircraft. An eyetrackerdevice monitors the eyes of a user and calculates the direction in whichthe user is looking and, in some applications, the particular point inthree dimensional space on which the user's eyes focus.

One commercial eyetracker is the Dual-Purkinje-Image (DPI) Eyetracker,manufactured by Forward Optical Technologies, Inc. of El Chaon, Calif.It determines the direction of gaze over a large two dimensional visualfield with great accuracy and without any attachments to the eye. Itoperates with infra-red light which is invisible to the subject and doesnot interfere with normal vision. The eyetracker has a pointing accuracyon the order of one minute of arc and response time on the order of onemillisecond. One can utilize the DPI Eyetracker with an infra-redoptometer to allow a continuous measure of eye focus, producing a threedimensional eyetracker.

The Problems

The prior art has failed to adequately address the need for controllingpower consumption in electrical devices having a finite source ofenergy, e.g. in battery driven devices such as computer displays. Insuch devices, power is frequently wasted by permitting the device tocontinue to run even though no user is in the vicinity. In the contextof a computer display, display power is certainly wasted if no one islooking at the display. In addition, in the prior art, when a screensaver switches on, and the screen suddenly goes black, a user, in thevicinity of the display, has his attention abruptly distracted towardthe screen which switched off.

There is thus a need for improving the control of power consumption inelectrical devices, particularly in computer displays. There is also aneed for improving the way in which screen savers are activated.

SUMMARY OF THE INVENTION

The present invention provides apparatus, processes, systems andcomputer program products which have the overcome the problems of theprior art. This is achieved by detecting when a user's attention isdirected to the electrical device and reducing the power consumptionwhen his attention is not so directed. It is also directed to detectingthe absence of a user in the vicinity of the electrical device andshutting down power consumption to an even greater level when thatoccurs. When a user returns, power is automatically reapplied.

The invention is directed to apparatus for automatically applying powerto an electrical device, including a motion detector, a proximitydetector and an optional infrared (IR) detector, activated by the motiondetector's detecting motion, and a switch connected to a source of powerand to the electrical device and controlled by the motion detector andthe proximity detector for applying power to the electrical device whenthe proximity detector detects an object within a predetermined distancefrom the electrical device while it is activated by the motion detector.A power off timer, activated when power is applied to the electricaldevice, is reset by the motion detector's detecting motion. It may beused for controlling the switch to remove power from the electricaldevice when the timer times out.

The invention is also directed to a computing device having a processor,a display having a controllable intensity, an eyetracker providing asignal indicating where a user's eyes are looking and a control forchanging intensity of the display based on that signal. The controlreduces the intensity gradually when a user looks away from the displayso as not to distract the user. When the user's eyes return to thedisplay for a predetermined period of time, the control reestablishesthe intensity level of the display in effect before the user lookedaway. The user's eyes returning to the display can be an instantaneousreturn, an return to the screen for a predetermined time interval orwhen the user's eyes fix on a particular point on the screen.Alternatively, the intensity level can be reestablished when the user'seyes begin to move toward the display. Power to the eyetracker isremoved when the user has not looked at the display for a predeterminedperiod of time.

The invention also relates to a method for automatically applying powerto an electrical device, by detecting motion, by detecting proximity ofobjects to the device; and by applying power to the device when anobject is within a predetermined distance from the electrical devicewithin a predetermined period of time after motion has been detected.

The invention also relates to a method of controlling intensity ofimages on a display, by detecting where a user's eyes are looking and bychanging intensity of images on the display based on where a user's eyesare looking.

The invention is also directed to a computer system including a network,a plurality of computers connected to the network, one of which is acomputer equipped with a motion detector a proximity detector activatedby the motion detector detecting-motion; and a switch connected to asource of power and to the electrical device and controlled by themotion detector and the proximity detector for applying power to the atleast a particular one of the plurality of computers when the proximitydetector detects an object within a predetermined distance from thecomputer.

The invention is also directed to computer program products eachincluding a memory medium and containing one or more computer programsand data used to implement the above methods, apparatus and systems.

Still other objects and advantages of the present invention will becomereadily apparent to those skilled in the art from the following detaileddescription, wherein only the preferred embodiment of the invention isshown and described, simply by way of illustration of the best modecontemplated of carrying out the invention. As will be realized, theinvention is capable of other and different embodiments, and its severaldetails are capable of modifications in various obvious respects, allwithout departing from the invention. Accordingly, the drawing anddescription are to be regarded as illustrative in nature, and not asrestrictive.

BRIEF DESCRIPTION OF DRAWINGS

The objects, features and advantages of the system of the presentinvention will be apparent from the following description in which:

FIG. 1 illustrates apparatus for automatically powering up and poweringdown an electrical device having an optional interface to a computerbus.

FIG. 2A is an illustration of a computer which is selectively batteryoperated and suitable for use with the invention.

FIG. 2B is an illustration of an exemplary computer architectureincorporating the invention.

FIG. 2C is an illustration of an exemplary memory medium used to storecomputer programs and data of the invention.

FIG. 3 is a state transition diagram of a computer process used inaccordance with the invention.

FIG. 4 is a state transition diagram of a power save process shown inFIG. 3.

FIG. 5 is a flow chart of a power down process shown in FIG. 3.

FIG. 6 is a flow chart of one power control process used as part of theinvention.

FIG. 7 is a flow chart of another power control process used inaccordance with the invention.

FIG. 8 is a flowchart illustrating another power control processconsistent with the invention.

NOTATIONS AND NOMENCLATURE

The detailed descriptions which follow may be presented in terms ofprogram procedures executed on a computer or network of computers. Theseprocedural descriptions and representations are the means used by thoseskilled in the art to most effectively convey the substance of theirwork to others skilled in the art.

A procedure is here, and generally, conceived to be a self-consistentsequence of steps leading to a desired result. These steps are thoserequiring physical manipulations of physical quantities. Usually, thoughnot necessarily, these quantities take the form of electrical ormagnetic signals capable of being stored, transferred, combined,compared, and otherwise manipulated. It proves convenient at times,principally for reasons of common usage, to refer to these signals asbits, values, elements, symbols, characters, terms, numbers, or thelike. It should be noted, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities.

Further, the manipulations performed are often referred to in terms,such as adding or comparing, which are commonly associated with mentaloperations performed by a human operator. No such capability of a humanoperator is necessary, or desirable in most cases, in any of theoperations described herein which form part of the present invention;the operations are machine operations. Useful machines for performingthe operation of the present invention include general purpose digitalcomputers or similar devices.

The present invention also relates to apparatus for performing theseoperations. This apparatus may be specially constructed for the requiredpurpose or it may comprise a general purpose computer as selectivelyactivated or reconfigured by a computer program stored in the computer.The procedures presented herein are not inherently related to aparticular computer or other apparatus. Various general purpose machinesmay be used with programs written in accordance with the teachingsherein, or it may prove more convenient to construct more specializedapparatus to perform the required method steps. The required structurefor a variety of these machines will appear from the description given.

DESCRIPTION OF THE PREFERRED EMBODIMENT

FIG. 1 depicts an apparatus for automatically powering up and poweringdown an electrical device in accordance with the invention. An automaticpower-up, power-down circuit 100 is shown optionally connected to bus180 over optional bus interface 175. The automatic power-up, power-downcircuit connects a source of power 105 to an output terminal 106powering the electrical device as described more hereinafter. Aself-powered motion detector 110, detects motion in the vicinity of theelectrical device. When motion is detected a bistable device 115 is setto close switch 120 to power-up proximity detector 125 and IR detector130. With these detectors powered up, if the motion detected by motiondetector 110 is within a certain proximity of the electrical device andif the amount of infra-red radiation emitted by the object detected isadequate, that is, above a certain threshold, both inputs to AND gate135 are activated, setting bistable 140 which then closes switch 145permitting power from source 105 to go to electrical device overterminal 106. With the application of power, power-down timer 150 beginstiming, counting down from a certain value. Any motion in the vicinityof the electrical device will trigger a motion detector periodicallyresulting in reset of the power-down timer. However, when a period oftime goes by with no motion detected, one may assume that a user hasleft the area and eventually power-down timer 150 will time out,resetting bistables 140 and 115, thus turning off power to theelectrical device at terminal 106 and turning off power to the proximitydetector 125 and the infra-red detector 130. As the user walks backtoward the electrical device, the motion detector will first sense thepresence and power-up proximity detector and infra-red detectors 125 and130 respectively and the cycle begins again.

Motion detector 110 can not distinguish between motion caused by a largeobject at a far distance or a small object at a close distance.Proximity detector 125 can distinguish how close an object is and alsowhether or not the object is closer than a particular threshold. Theinfra-red detector detects whether or not the moving object within acertain proximity is a living object or not, and, based on the amount ofinfra-red radiation, can determine the approximate size of the object.By using all three of these detectors, one may ensure that a humanoperator is close enough to the electrical device to want to use it.Under those conditions, the electrical device is powered-up.

The optional bus interface 175 and the optional computer bus 180 areused as discussed hereinafter for activating computer display screens.

The infra-red detector can also be utilized to distinguish the situationin which the computer is being carried by a person from one in which amotion results from a person approaching. If a person is approaching,the IR intensity will be increasing, whereas if the device is beingcarried, the IR levels will remain constant.

FIG. 2A is an illustration of a computer which is selectively batterypowered and suitable for use with the invention. The illustration of thecomputer corresponds to any one of a variety of standard battery poweredportable computers 200. Such computers typically have a keyboard 210which is exposed when open, a disc drive 215, a mouse 220, which may beincorporated into the keyboard, and a display 225 for displaying outputfrom the processor. In one embodiment, the display is a touchscreendisplay. In accordance with the invention, an eyetracker sensor is shownat 230, positioned so as to be able to view the user's eyes. The use ofthe eyetracker sensor will be described more hereinafter.

FIG. 2B is a block diagram of the internal hardware of the computer ofFIG. 2A. A bus 250 serves as the main information highwayinterconnecting the other components of the computer. CPU 255 is thecentral processing unit of the system, performing calculations and logicoperations required to execute a program. Read only memory (260) andrandom access memory (265) constitute the main memory of the computer.Disk controller 270 interfaces one or more disk drives to the system bus250. These disk drives may be floppy disk drives, such as 273, internalor external hard drives, such as 272, or CD ROM or DVD (Digital VideoDisks) drives such as 271. A display interface 275 interfaces display220 and permits information from the bus to be displayed on the display.Communications with external devices can occur over communications port285.

An automatic power-up/power-down circuit 100 is connected to the bus 250over bus interface 175. Power from power source 105 is utilized topower-up the computer and the bus structure over terminal 106. Theoutputs from the motion detector, proximity detector and IR detector ofthe automatic power-up/power-down circuit 100 are connected to the busand are utilized as more fully described hereinafter.

An eyetracker 290 is interfaced to the bus over interface 289 andprovides information for control of the power and described morehereinafter. The display 220 is interfaced to the computer bus overdisplay interface 275. A separate control line 276 is shown between thedisplay interface 275 and the display 220. This line is utilized tocontrol the intensity of illumination of images on the surface of thedisplay. It effectively serves as a power control for the displaydevice.

FIG. 2C illustrates an exemplary memory medium which can be used withdrives such as 273 in FIG. 2B or 210A in FIG. 2A. Typically, memorymedia such as a floppy disk, or a CD ROM, or a Digital Video Disk willcontain, inter alia, program information for controlling the computer toenable the computer to perform its testing and development functions inaccordance with the invention.

FIG. 3 is a state transition diagram showing the control processes usedin accordance with the invention. The process begins with a power savestate 310 which is described more in detail in FIG. 4. From the powersave state, the state can transition either to a power-up state 320 orreturn to itself. From power-up state 320, the invention can transitionto a power-down state shown more in detail in FIG. 5 (330) or return toitself.

Turning to FIG. 4, if motion detector 110 shown in FIG. 1 detects motion(410), the proximity detector and the IR detector are activated (420).If they are both activated, then a check is made to determine ifproximity of the object whose motion is detected is less than thethreshold (430) and then a check made to see if the IR level is greaterthan a threshold (440). If it is, switch 145 shown in FIG. 1 is closedand power is applied to terminal 106 to power-up the external devicethus entering the power-up state 320 shown in FIG. 3. States 430 and 440can transition to “set timer” state 450 if their conditions are not met.After timer 450 times out, it will transition to state 460 where theproximity detector and the IR detectors will be deactivated. State 460will transition back to state 410 and the process begin again. State 460may also be entered externally from the power-down state 330 shown inFIG. 3.

FIG. 5 shows more in detail the power-down state transition diagram 330of FIG. 3. When entered from the power-up state 320 of FIG. 3, a settimer state 520 is entered which corresponds to power-down timer 150shown in FIG. 1. If motion is detected (state 530) timer 520 is reset.If no motion is detected, state 540 results from a timeout whichtriggers a power-down device state 550. This corresponds to resetting offlip-flops 140 and 115 if FIG. 1. State 550 transitions back to powersave state 300 shown in FIG. 3 and more specifically to state 460 withinthat state.

FIG. 6 is a flow chart of a one power control process used as part ofthe invention. Eyetracker 290, shown in FIG. 2B is utilized to controlthe illumination of images on the display 220. How this is done is shownin FIG. 6. The eyetracker outputs are processed to distinguish fourconditions shown in FIG. 6, namely:

1. Whether the eyes are fixed at a point on the screen,

2. Whether the eyes move off the screen,

3. Whether the eyes are approaching the screen from a position off thescreen, and

4. Whether the eyes are moving across the screen.

These four cases are distinguished by separate processing branches shownin FIG. 6. When the eyetracker determines that the eyes are fixed on thescreen, case 1 (610) obtains and the display intensity is set at normalillumination (615).

In case number 2 (620), when the eyes move from the screen to a pointoff the screen, a time interval of, preferably, {fraction (1/10)} of asecond (625) is set. If that time expires without the eyes returning tothe screen, the screen will slowly fade the display intensity to black(626). In the embodiment shown in FIG. 6, once the eyes have been offthe screen for a period of time greater than the time set in item 625,cases 3 and 4 are treated identically. That is, whether the eyes areapproaching the screen or moving across the screen without fixing on thescreen, the display intensity will resume normal illumination as quicklyas possible. Normal illumination will thus continue until such time asthe eyes leave the screen again.

In one embodiment, step 626 includes steps shown in FIG. 8. In step 830,power to the eyetracker is removed when it is determined in step 827that the user has not looked at the display for a predetermined periodof time. In step 840, power to the SEP eyetracker is restored after theuser has not looked at the display for a predetermined period of time bya user input. If the display is a touchscreen display, the user input isprovided by touching the touchscreen display.

The embodiment shown in FIG. 7, is identical for cases 1 and 2 as thatshown in FIG. 6. However, cases 3 and 4 are treated separately. In case3, where the eyes are approaching the on-screen condition, in thisembodiment, nothing happens. That is, the screen remains blank. However,case 4 results in measurement of the time that the eyes are on thescreen. If the time the eyes are on the screen exceeds some threshold,the display intensity is resumed at normal illumination as quickly aspossible.

Thus, in accordance with the invention, electrical devices powered byenergy sources of finite capacity can utilize the energy available tothe maximum extent possible and reduce energy waste to a minimum.

In this disclosure, there is shown and described only the preferredembodiment of the invention, but, as aforementioned, it is to beunderstood that the invention is capable of use in various othercombinations and environments and is capable of changes or modificationswithin the scope of the inventive concept as expressed herein.

What is claimed is:
 1. A computing device comprising: a processor; adisplay having a controllable intensity connected to said processor; aneyetracker providing a signal indicating where a user's eyes are lookingto said processor; and a control for changing intensity of said displaybased on said signal, wherein the display is a touchscreen display, andwherein power to the eyetracker is removed when the user has not lookedat the display for a predetermined period of time and power to theeyetracker is restored by touching said touchscreen display.
 2. Acomputing device comprising: a processor; a display having acontrollable intensity connected to said processor; an eyetrackerproviding a signal indicating where a user's eyes are looking to saidprocessor; a control for changing the intensity of said display based onsaid signal, a motion detector; and a proximity detector, activated bysaid motion detector detecting motion; wherein power to the eyetrackeris restored when the proximity detector detects an object within apredetermined distance from the computing device while activated by saidmotion detector.